A useful technique for piecewise deterministic Markov decision processes
نویسندگان
چکیده
This note presents a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. produces an auxiliary PDMDP from original one. The possesses certain desired properties, which may not be possessed by PDMDP. We apply this to risk-sensitive PDMDPs total cost criteria, comment on its connection uniformization technique.
منابع مشابه
Demographic noise and piecewise deterministic Markov processes.
We explore a class of hybrid (piecewise deterministic) systems characterized by a large number of individuals inhabiting an environment whose state is described by a set of continuous variables. We use analytical and numerical methods from nonequilibrium statistical mechanics to study the influence that intrinsic noise has on the qualitative behavior of the system. We discuss the application of...
متن کاملPiecewise deterministic Markov processes in biological models
We present a short introduction into the framework of piecewise deterministic Markov processes. We illustrate the abstract mathematical setting with a series of examples related to dispersal of biological systems, cell cycle models, gene expression, physiologically structured populations, as well as neural activity. General results concerning asymptotic properties of stochastic semigroups induc...
متن کاملA finite difference method for piecewise deterministic Markov processes
An extension of non-deterministic processes driven by the random telegraph signal is introduced in the framework of piecewise deterministic Markov processes [9], including a broader category of random systems. The corresponding Liouville-Master Equation is established and the upwind method is applied to numerical calculation of the distribution function. The convergence of the numerical solutio...
متن کاملPiecewise Deterministic Markov Processes for Continuous-Time Monte Carlo
Recently there have been conceptually new developments in Monte Carlo methods through the introduction of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has led to some fundamentally new Monte Carlo algorithms which can be used to sample from, say, a posterior distribution. Interestingly, continuous-time...
متن کاملNumerical method for optimal stopping of piecewise deterministic Markov processes
We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location – inter-arrival time Markov chain naturally embedded in the PDMP, and path-adapted time discretization grids. It allows us to derive bounds for the convergence rate of the algorithm an...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Operations Research Letters
سال: 2021
ISSN: ['0167-6377', '1872-7468']
DOI: https://doi.org/10.1016/j.orl.2020.11.002